Rashomon

Use betting market mechanisms to quantify bias in the news media, and offer news organizations an alternative business model in an effort to curtail fake news. ¹ A demo version is live at rashomonnews.com.

The problem

Fake news is prevalent because the incentives are misaligned. Consumers are rewarded for becoming more tribal, more polarized, and for seeking out what confirms their worldview. Media organizations are desperate enough to go along with it because it generates enough ad revenue to get by. Neither side is incentivized to figure out the truth.

Most consumers would say they follow the news to stay informed. In practice, people keep up with the news to:

1. Gather evidence for their worldview, which produces confirmation bias.
2. Show people they admire that they, too, are smart, which produces filter bubbles.
3. Signal to those same people that they share common ground, which produces echo chambers.

Media companies would say their incentive is a civic duty to shine a light on the ills of society. Really, their incentives are:

1. Survival. Print doesn’t sell, and competitors give content away for free, setting a market price of $0.
2. Profit. That means paywalls or advertising, and chasing eyeballs for ads pushes papers toward viral techniques and sub-par journalism.
3. Looking like they have integrity. Not cynically: papers need to be trusted to convert that trust into subscriptions, so they have to earn it, or at least appear to.

The proposal

Ideally I would pay everyone to read everything equally: Fox, CNN, Infowars, HuffPost, all of it. The only thing that overcomes the urge to chase the news you want is being paid to give a fair chance to the news you don’t. I’m not remotely wealthy enough to do that.

The next best thing is the opportunity to get paid for reading something you might disagree with, and the closest mechanism to that is a bet. Betting forces what Nassim Nicholas Taleb calls skin in the game. You become much more hesitant to call something fake unless you’re sure.

For the media organizations, the answer is a betting exchange that earns a commission on each bet. An exchange sets up the bet, maintains fairer odds than a bookie, and, if run to share that commission with news companies, could replace the clickbait, ad-driven model that’s eroding the idea of truth.

In practice: you receive an anonymized, randomized article with no giveaways to outlet or author, and at the end you decide where it falls. Democratic or Republican, sponsored or genuine, real or satire. ² Everyone disputes the truth of an event, but it’s much harder to dispute a publication’s leaning relative to its peers. Few would argue Fox News sits as far left as Antifa News. This is sometimes loosely called the Overton window. The question isn’t whether the article is true, but where it sits on a relative scale.

AllSides already publishes relative ratings using a similar method, with evaluators of stated bias reading anonymized articles. Those ratings are a good starting ground truth, and a voting mechanism can come later. ³ Which lands close to Robin Hanson’s mantra: “We vote on values, we bet on beliefs.”

With the system sketched out, I built a demo in Python and Django on PostgreSQL and Redis, with a minimal Bootstrap front end. A large part of the project was learning the fundamentals of web development.

Analysis

The demo was alpha-tested during the 2020 election by 72 users, mostly friends and family recruited through social media. The sample is sparse, but its timing makes it unusual. ⁴ Each account was preloaded with $10 of play money. Of the $720 available, $217.20 was wagered, 30.2% utilization, across 223 bets.

There were more incorrect bets (125) than correct ones (98), but people seemed to sense when they were wrong: $101.26 was wagered on correct bets against $115.94 on incorrect ones. Treating stake as a proxy for confidence, users bet less when they suspected they were guessing.

Grouping bets by source bias is more useful. By amount wagered, users got around 50% correct on left, center, right and far-right sources, well above the 20% expected from random picks across five categories. More money was bet incorrectly on far-left sources, suggesting overconfidence there.

By count, users were most comfortable with left-leaning sources, correct 52% of the time, and least accurate on far-right sources, around 25%. They identified far-left sources fairly well by count but couldn’t capitalize by amount, which points to unusually low confidence on the far left.

Incorrect bets can also be read as a direction of error. Calling a center source left-leaning implies the bettor leans right, and vice versa. The data shows a weak version of this. By count, left-wing sources erred rightward while center and right-wing sources erred leftward. ⁵ Far-left and far-right sources can only err in one direction, since there’s no option beyond either end. Either users are already polarized toward the extremes, or they are pulling every source toward the center.

What stands out is how abruptly the direction of error flips from mostly rightward to mostly leftward.

Further plans

Development has stalled, but the idea is flexible. A full platform could add its own currency, social sharing after bets settle, a terminal-style interface for tracking positions, and integration with prediction markets like PredictIt or Polymarket. ⁶ The pipe dream: combine Peter Turchin’s cliodynamics, news betting, and prediction markets into models of past, present and future that feed one another.

It extends beyond the left–right axis, too: sponsored versus genuine, or new games like spread betting, betting against the algorithm, against other users, or against the house. There may even be analogues to options markets.

Technically, the demo is bare-bones. A mobile app via React Native is the obvious next step, along with fixing authentication (there’s no password reset) and article images that sometimes give away the source.

Conclusion

Rashomon never reached enough users for statistically significant results, but the data it did collect is a snapshot of the tumultuous 2020 election. It was a reasonably original idea, and building it was a tremendous learning opportunity. I wish I could have taken it further, and I’m grateful for how far it got.